feat(gateways): Kafka Gateway add DescribeConfigs and AlterConfigs APIs by ryerraguntla · Pull Request #4398 · apache/iggy

ryerraguntla · 2026-10-04T13:08:21Z

Which issue does this PR address?

Partially closes #3546 (Part 2)
Relates to #3560

Rationale

Kafka admin clients read and change topic retention with DescribeConfigs (32) and the deprecated AlterConfigs (33). Iggy already stores that lifetime as message_expiry, and retention applies to sealed segments only. The active segment does not expire. CreateTopics rejects a per-topic config list and tells the client to set retention.ms with AlterConfigs, so these two APIs are how a client reads and writes that expiry after the topic exists.

This PR covers the topic-config half only. Unknown keys are INVALID_CONFIG (40). Kafka has no NOT_CONFIGURABLE code, which is the code the issue text suggested.

What changed?

With the bridge on, DescribeConfigs (versions 1-4) answers topic resources only. It returns retention.ms from Iggy message_expiry (-1 when sealed segments never expire) and a read-only cleanup.policy=delete. AlterConfigs (versions 0-2) persists only retention.ms, through update_topic. A value whose seconds exceed u32::MAX is rejected and is not stored. validate_only runs the same checks, including the topic lookup, and does not write.

One bad key fails that resource only. Other resources in the same request are still processed. With the bridge off, each resource gets NOT_CONTROLLER (41) and nothing is read or written. A missing topic is UNKNOWN_TOPIC_OR_PARTITION (3), checked before the config keys. More than 100 distinct topics returns POLICY_VIOLATION (44) on every resource in the request.

IncrementalAlterConfigs (44) is not implemented and is not advertised. A client that sends it gets the connection closed. Only the deprecated Admin.alterConfigs (33) writes retention.ms. CreateTopics still rejects per-topic configs and tells the client to use AlterConfigs. DeleteTopics is not in this PR and is covered in a separate PR. IncrementalAlterConfigs (44) will be implemented in another succeding PRs.

Local Execution

Passed:

cargo build -p iggy-gateway-kafka --tests
cargo clippy -p iggy-gateway-kafka --all-features --all-targets -- -D warnings
Unit tests for topic config, bounds, config handlers, version firewall, CreateTopics, and kafka-message-gen
Kafka 3.9 Java AdminClient probe on the host JDK: retention.ms=-1, then alter to 8000, unknown key rejected, validate_only did not write, describe_batch_50_elapsed_ms=22. The probe prints that elapsed time and does not assert a latency threshold.

AI Usage

Fill this in before opening.

Which tools? Claude
Scope of usage? Implementation and review
How did you verify the generated code works correctly? Walk through and manual testing.
Can you explain every line of the code if asked? Yes

…n clients need to read and set topic retention. DescribeConfigs reports retention.ms from Iggy message expiry and a read-only cleanup.policy. AlterConfigs persists only retention.ms, capped so segment expiry cannot wrap.

…n clients need to read and set topic retention. DescribeConfigs reports retention.ms from Iggy message expiry and a read-only cleanup.policy.AlterConfigs persists only retention.ms, capped so segment expiry cannot wrap.

gateways/kafka/docs/CONFIGS.md gateways/kafka/src/protocol/handlers/alter_configs.rs gateways/kafka/src/protocol/handlers/describe_configs.rs gateways/kafka/src/protocol/handlers/topic_config.rs gateways/kafka/tests/common/wire.rs gateways/kafka/tests/configs_handler_tests.rs gateways/kafka/tests/configs_java_client_tests.rs gateways/kafka/tests/configs_real_bridge_tests.rs

ryerraguntla · 2026-10-04T13:09:23Z

Keeping it in draft mode as there few edge cases needs to be worked and documented

Milliseconds stay the value stored in Iggy. Those names are remembered as synonyms only when an alter uses them.

codecov · 2026-10-04T14:06:16Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.45%. Comparing base (0918395) to head (083600f).
⚠️ Report is 3 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #4398      +/-   ##
============================================
+ Coverage     88.43%   88.45%   +0.02%     
+ Complexity     1607     1579      -28     
============================================
  Files          1302     1293       -9     
  Lines        252565   248952    -3613     
  Branches     212730   212367     -363     
============================================
- Hits         223358   220217    -3141     
+ Misses        23811    23537     -274     
+ Partials       5396     5198     -198

Components	Coverage Δ
Rust Core	`89.56% <73.33%> (+<0.01%)`	⬆️
Java SDK	`68.71% <ø> (-0.09%)`	⬇️
C# SDK	`77.60% <ø> (-1.15%)`	⬇️
Python SDK	`91.16% <ø> (+0.09%)`	⬆️
PHP SDK	`85.67% <ø> (ø)`
Node SDK	`96.59% <ø> (+0.08%)`	⬆️
Go SDK	`70.19% <ø> (-0.06%)`	⬇️
C++ SDK	`∅ <ø> (∅)`
see 100 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

ryerraguntla · 2026-10-04T20:39:33Z

/skill team-review-slim

github-actions

Summary: The new Kafka DescribeConfigs and AlterConfigs handlers carry three client-visible defects: per-resource Iggy round trips that serialize on the shared lockstep client, an unmodeled describe key that fails the whole resource, and an empty AlterConfigs list answered as success. Two test comments are stale, and two simplifications are noted.

Counts: critical 0, warning 3, nit 2, simplification 2

This review was generated by Claude Code 2.1.284 on deepseek-flash[1m]. Review the output before you act on it.

ryerraguntla · 2026-10-04T22:00:09Z

/ready

krishvishal

Main issues: the timeout comment in alter_configs.rs does not match the code, repeated describe keys blow up the response, and retention.minutes/hours are not Kafka topic configs.
Details inline. The simplification comments are cleanups, not blockers.

ryerraguntla · 2026-10-08T01:14:49Z

/ready

ryerraguntla added 3 commits October 3, 2026 23:43

github-actions Bot added the S-waiting-on-review PR is waiting on a reviewer label Oct 4, 2026

ryerraguntla marked this pull request as draft October 4, 2026 13:08

github-actions Bot removed the S-waiting-on-review PR is waiting on a reviewer label Oct 4, 2026

feat(gateways): accept retention.minutes and retention.hours

e830007

Milliseconds stay the value stored in Iggy. Those names are remembered as synonyms only when an alter uses them.

feat(gateways): fix the wire fixures.sh

927bfe6

ryerraguntla marked this pull request as ready for review October 4, 2026 20:41

github-actions Bot added the S-waiting-on-review PR is waiting on a reviewer label Oct 4, 2026

github-actions Bot reviewed Oct 4, 2026

View reviewed changes

github-actions Bot added S-waiting-on-author PR is waiting on author response and removed S-waiting-on-review PR is waiting on a reviewer labels Oct 4, 2026

Fixing team review skill

42ec19c

github-actions Bot added S-waiting-on-review PR is waiting on a reviewer and removed S-waiting-on-author PR is waiting on author response labels Oct 4, 2026

krishvishal requested changes Oct 6, 2026

View reviewed changes

github-actions Bot added S-waiting-on-author PR is waiting on author response and removed S-waiting-on-review PR is waiting on a reviewer labels Oct 6, 2026

numinnex reviewed Oct 6, 2026

View reviewed changes

ryerraguntla added 6 commits October 7, 2026 05:23

Fixing review comments

f6eacae

Fixing Doc tests

c4afa1b

Fixing mac clippy tests

9229587

Fixing fmt error

1f644fa

Fixing clippy and fmt error

41134ed

Fixing clippy and fmt error

6f8a927

github-actions Bot added S-waiting-on-review PR is waiting on a reviewer and removed S-waiting-on-author PR is waiting on author response labels Oct 8, 2026

ryerraguntla and others added 2 commits October 8, 2026 11:19

Merge branch 'master' into feat/gateways-3546-describe-alter-configs

afa9b80

Fixing the merge issues

ca7fa81

krishvishal approved these changes Oct 9, 2026

View reviewed changes

Merge branch 'master' into feat/gateways-3546-describe-alter-configs

606dcf9

hubcio approved these changes Oct 10, 2026

View reviewed changes

Merge branch 'master' into feat/gateways-3546-describe-alter-configs

083600f

hubcio merged commit 5f34dbc into master Oct 10, 2026
69 checks passed

hubcio deleted the feat/gateways-3546-describe-alter-configs branch October 10, 2026 17:12

github-actions Bot removed the S-waiting-on-review PR is waiting on a reviewer label Oct 10, 2026

This was referenced Oct 13, 2026

Kafka gateway: accept the topic configurations that frameworks send #4508

Open

Kafka gateway: serve IncrementalAlterConfigs (key 44) #4509

Closed

fix(gateways): stop reporting cleanup.policy and accept -1 defaults #4534

Merged

Conversation

ryerraguntla commented Oct 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Which issue does this PR address?

Rationale

What changed?

Local Execution

AI Usage

Uh oh!

ryerraguntla commented Oct 4, 2026

Uh oh!

codecov Bot commented Oct 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

ryerraguntla commented Oct 4, 2026

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ryerraguntla commented Oct 4, 2026

Uh oh!

krishvishal left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ryerraguntla commented Oct 8, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ryerraguntla commented Oct 4, 2026 •

edited

Loading

codecov Bot commented Oct 4, 2026 •

edited

Loading